This invention relates to a solid state image pickup apparatus or solid-state television camera having the function of compensating for image defects. Signal processing is used for compensating for a deterioration in picture quality due to an image pickup output from a defective picture element in a solid state image pickup device, such as a charge coupled device (CCD).
Generally, in a solid state image pickup apparatus provided with an image pickup unit composed of solid state image pickup devices, the image pickup output is obtained at the solid state image pickup devices using a field readout mode or a frame readout mode. In the field readout mode signal charges are read out from the totality of the picture elements or pixels during one field period, whereas in the frame readout mode the signal charges are read out from the totality of the pixels during one frame period. Additionally, an electronic shutter function is performed by controlling the effective charge accumulation time of the solid state image pickup device. The image pickup unit of a solid state color image pickup apparatus adapted for color image pickup performs image pickup with three solid state image pickup devices. Color video signals are formed from the image pickup outputs of scene images formed by image pickup light color-separated into the three prime colors of red (R), green (G) and blue (B). For increasing the resolution in the horizontal direction, a space pixel shifting system is used in the solid-state color image pickup apparatus for imaging a scene or an object for each of the color components of the imaging light. Accordingly, a solid-state image pickup device, for example, the solid-state image pickup device adapted to image the scene of a green (G) color component, is arranged at a position shifted by one half pixel pitch in the horizontal direction with respect to the other solid-state image pickup devices, for example those adapted to image the scenes of the red (R) and blue (B) color components.
It is noted that, in the solid-state image pickup device formed by semiconductors, such as CCDs, defective pixel representations are inevitably produced in which, due to the local crystal defects of the semiconductors, a constant bias voltage is added to the image pickup output in proportion to the incident light volume. As a result, the picture quality is deteriorated. The image defect consisting of a constant bias voltage always being added to the image pickup output is called the white flaw defect, since the image defect signal, if processed directly, will appear as bright spots on the monitor screen surface.
To compensate for the deterioration in the picture quality due to the defective pixel representation by the solid-state image pickup device, the conventional practice is to store the information indicating the presence or absence of the pixel defect in memory and to use the imaging output from the pixel adjacent to the defective pixel for interpolating an imaging output to be used in place of the imaging output from the defective pixel.
When there is a defect in the solid-state image pickup device for each pixel in the memory, it becomes necessary to make use of a memory having a voluminous capacity corresponding to the total number of the pixels of the image pickup device. In such case, it has been proposed in our senior patent application, now matured to the Japanese Patent Publication KOKOKU No. 34872/1985, to encode and store the distance between the defective pixels in memory to indicate the position of the defective pixel contained in the solid-state image pickup device, thereby reducing the overall memory capacity.
Although the memory capacity can be reduced by encoding and storing the distance between the defective pixels, the memory is in perpetual operation, resulting in an increase in wasteful power consumption and an increase in the power consumption of the image pickup apparatus as a whole.
On the other hand, the above described compensation by interpolation leads to increased compensation error there is no correlation to the image pickup output for the pixel lying close to the defective pixel. In such case it has been proposed (1) to store the data concerning the position of the defective pixel and the output signal level of the defective component in the memory and (2) to form defect compensation signals at the timing of the output signal of the defective pixel contained in the output signal from the solid state device. Such defect compensation signals are added to the output signal of the device for realizing the compensation defects, (see the Japanese Patent Application KOKAI No. 51378/1985).
It is noted that, in the conventional solid state imaging or image pickup apparatus, defect compensation is made only for image defects caused by the frequent white flaw image defects.
It is noted that, in the solid state imaging device formed by semiconductors, the signal level due to the spurious signal charges and ascribable to the dark current is high. As a result, the image defect caused by the white flaw image defect presents itself rather acutely. However observation of image defects when suppressing the dark current to a lower value has revealed that, besides the white flaw image defect so far known to have a temperature dependency, black flaw image defects which are free from temperature dependency but in which predetermined bias charges are substracted from the image pickup output in proportion to the incident light also are present. In addition, white and black flaw image defects free from temperature dependency but having an incident light volume dependency appear as image defects in the imaging output.
The image defect caused by the white flaw appears rather acutely, but at an extremely low level at an ambient temperature, so that it may be safely disregarded. However, the image defect increases exponentially with increases in temperature. To compensate for the white flaw defects having the temperature dependency, the compensation signal needs to be compensated further for temperature effects. However, should there be any compensation error in the temperature compensating circuit, the white flaw defect may remain overcompensated or under compensated such that so-called compensation flaws are left over in the compensated image pickup output.
In the solid state imaging apparatus performing an electronic shutter function, the charge accumulation time of the solid state imaging device of the image pickup unit is variably controlled by the present speed of the electronic shutter causing a change in the defect level due to defective pixels. The defect level also is changed by switching the readout modes of the signal charges. In case of switching the readout modes for the signal charges, if the charge accumulation time is set to 1/2 in, for example, the field readout mode, the produced signal charges are equal to one half those for the normal mode. However, for the frame read-out mode, if the charge accumulation time is set to 1/2 the effective charge accumulation time is one fourth that for the normal accumulation time. Even when the shutter speed is set to the same value, the effective charge accumulation time will differ depending on the readout mode for the signal charges, so that the signal level of the white flaw defect signal included in the image pickup output will also differ. As a result, when the charge accumulation time for the solid state imaging device is changed, an error is caused in the white flaw defect compensation process and a so-called compensation flaw is left over in the image pickup output already compensated for defects.
When the data concerning the positions of the defective pixels included in the solid-state image pickup device are stored in the memory and compensation is made on the basis of the data read out from the memory, compensation for defects can be made very satisfactorily without concomitant compensation errors. However, the volume of the data is increases undesirably since the data concerning the location of the defective pixels and the level of the defective components included in the output signal need to be stored in the memory. When the location of each of the defective pixels is indicated by an absolute address, the number of bits used as the position data for each defective pixel increases for a solid state image pickup device which has an increased number of pixels to increase the high image resolution. Thus, the memory capacity is necessarily increased.
When the data concerning the position of the defective pixel and the level of the defective component output signal are stored in the memory, a defect compensation signal is formed to be added to the output signal of the solid state image pickup device should the position of the defect compensation signal be deviated with respect to the output signal of the solid state image pickup device, so-called compensation flaws occur due to the defective compensating operation and deteriorate the quality of the image pickup output signal. In addition, should it be intended to compensate for the defects in such a manner as to take the above described changes in the various defect levels into account, it is necessary to carry out an extremely complex processing operation. In the solid state image pickup apparatus having an image pickup unit constructed in accordance with the spatial pixel shifting system, it is necessary to form defect compensating signals in phase with the image pickup outputs obtained at each of the first, second and third solid state image pickup devices.